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(54) Method for automatically mapping of geometric objects. in digital medical images 

(57) A method of mapping geometrical objects to a 

digital image. At least one anchor point in a model is. . f^-f ■ 

selected and mapped to the digital image. A geometric 
transfomiation is computed on tiie basis of at least ona 
corrsspondence betyveen an anchor object and a corre- 
sponding mapped object, and the calculated geometric 
transfomiation is applied to at least one geometric ob- 
ject thereby mapping it from the model image to a cpr- 
responding mapped position in the digital image. 



— 



Taiijet image { 
wrthmapped | 
ejects rendered | 



a. 
lit 



EP 1 598 778 A1 



Description r^,-..— 
i=1ELD OF THE INVENTION 

[0001] The present invention relates to computer-assisted method to map geometric objects in a digital medical 
image. 

BACKGROUND OF THE INVENTION 

[0002] In radiological practice, the radiographer frequently needs to locate specific anatomic structures on the radi- 
ograph in order to focus his attention on these specific image areas. The position of the image areas will typically 
depend on the examination type of the radiograph and the suspected disease; hence these locations are examination 
and diagnosis-specific. 

[0003] For example, in bone age assessment according to the Tanner-Whitehouse method, 20 bony regions of iri- 
teresl (ROI) need be located on a hand radiograph, and staged against a set: of reference stages depicted in an atlas. 
In European patent application EP A 1 .293 &25, a method is disclosed to automate ttie staging and scoring of these 
skeletal sites. The associated ROI of these sites Is extracted and compared against a set of reference pictures and 
diagrams by presenting a given ROI simultaneously In close vicinity with its reference set. The location process for 
each of these riegions of Interest must be perfomnied irrespective of thepositlon of the hand itself in the image. A typical 
exposure error introducing positional variation is that the hand axis is placed in a rotated fashion with respect to the 
cassette borders of a cassette conveying a radiation detector such as a photo-stimulable phosphor screen on which 
the radiation image is recorded. Furthermore, irregular finger spread of fingers adds a second degree of rotational 
variation in the location of these ROI's. Hence, there is a need to enhance the location and extraction process of ROIs 
in a medical image. 

{0004} A further location determination task lathe Tanner and Wfiitehouse scoring method Is that anatonilc bony 
featured are compared against geometric ob/eets in relationship to anatomy, Forexample, irv the scoring of the scaphoid, 
the stages G and H are characterized in that the direction of the lunate bdreJerof the scaphoid is compared with respect 
to the rnldllne of the capitate. Therefore, the radiographer must continuously mentalfy map the capitate midline into 
. the image in order to verify this feature and assign the correct stage to the scaphoid. Hence, there is a need to map 
■ ■ this diagnostic line into the actual image, so as tb avoid the need of continuous mental relocation. 
[0005] A second diagnostic activity where objects need be located in an Image is the field of geometric measure- 
ments . Obvio usiy. in the film-based measurenjents, no assistance is offered in drawing the geometric objects or entities 
in the image. In European patent application EP A 1 a49 098 a method is disdosed to automate the measurements in 
digitally acquired medical images by grouping measurement objects and entities into a computerized measurement , 
scheme consisting of a bi-directionally linked external graphical model and an internal informatics model. In a meas- 
urement session according to European patent application EP A 1 349 098, a measurement scheme is retrieved from 
the computer and activated. Measurements are subsequently performed on the displayed image under guidance of 
the activated measur^nliferif scheme. 

[0006] : In this computerized method, a multitude of geometric objects must be mapped in the digital image onto which 
other measurement obje<^ and finally measurement entities (such as distances and an^es) are based. The basic 
geometric objacte-are typical^ key user points, which define other geometric objects onto which measurements are 
based. Forexample, iwo pairs of key pdnts each define a I6ie, and an angle between the resulting line pair, representing 
the ar^utatiori between anatomic structures,- is computed. These key user points must be mapped in the image onto 
their correspondinganatomical fandmark positions! In order to facilitate the initial position of these key user points, 
thelrpbsltlon may be computed on the basis of a small number of anchor points, which are also defined in relationship 
with anatomy and will typically coincide with easily discernible anatomical landman. 

[0007] A further Ipcation task when performing measurements in digital images is related to the selection of the image 
area and. the proper resolution of the image for placing the geometrical objects sueh.as measurement points. For large 
images of which only a downscaled version Is displayed, the radiographer will usually prefer to position the key user 
points in the origindf image in order to use the full resolution for better positional accuracy of the key user points. 
Therefore, the image area around each sequertfial i$cy user point must be dispi^ed in ite original resolution, which 
requires that the user first select the image area to be magnified. This Is typteally done-by mettiode of the prior art by 
an additional mouse click in ttie image in the vicinity oi the desired key user poivi, upon whk:h the image is displayed 
at full resolution, orby pantiing in a second image window di^aying the image at full resolution iintil the region around 
the desired key user point is viewed. However, the drawtjack of this manner of operation is that when the number of 
key user points is large, which is typically the case for complex measurement schemes, a large number of mouse 
manipulations is needed, hence this melhbd is slow, fatiguing and error prone. 

{OOOSt A thifd area itt: radiological Qperation that also Fequires a geometric location is that of quaTity assurance and 
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quality control by means of an analyzing phantom. For example, in rhammographic system qualifyrcantioti a phantom 
with a specific spatial arrangement of geometric structures is used. These strucaures such as disk and bars serve to 
determine the image contrast and compare it against perceptibility thresholds. Geometric accuracy may be checked 
by measuring the structures' spatial locations in the imageandcompare them with theirmodel counterpart. To automate 
the analysis of the imaged phantom, a CAD model may be used to reference the geometric structures against a coor- 
dinate reference system. This reference system may be established by means of small number of control points of 
known location. For these control points are part of the phantom, (hey are imaged simultaneously with the analyzing 
objects; By locating their position in the inrage, a correspondence between model and actual imageean be established. 
After establishing the geometric transformation and perfonning the mapping of the modei structures, as will be detailed 
in the sequel, the initial position of all referenced objects is easily found back in the image, and subsequent quality 
parameters are computed. 

SUMK4ARY OF THE INVENTION 

[0009] To overcome, the drawbacks of the prior art the present invention provides a method as set out in the claims. 
[0D10] The present invention discloses a model-based geometric mapping method. In particular, geometric objects 
that need to he mapped in an image are encoded relative to a set of model anchor geometric objects, wtiich in the 
preferred embodiment are key user points. The user manually maps the model anchor objects In the Image, the number 
of which-is typically very small. A geometric mapping transformatk>n is subsequently established between the model . 
anchbr objects and the user-mapped model anchor objects. Subsequently, all geometric objects that need to be mapped 
in thCimage and that are specified within a coordinate system deftned by the mapping model anchor objects, are 
autbmatically positioned in the target image, tiy applying the computed geometrictransformation to the objects' defining 
. parameters. Finally all mapped, objects are rendered in the target inriage according to the values of their def ining pa- 
rameters. ■ . , 

Ch&mcterisiicsaf model anchor, objects 

[OOttJ The characteristics of model anchor objects can be enumerated as follows: ■: 

• ■ ■ Although any object may be used as a model anchor objects to whteh the geometric objects are defined, the rnodet 

anchor obje<^ will typically be easy to position, hence their defining number of degrees of freedom v/ilLbe small. 
The number of degrees of freedom (d.o.f.) is the number of independent position variables that have to be.spqcified 
in order to locate all parts of an object. Eligible anchor objects are for example anchorpolnts (2 d.o.f. (displacement 
in X and y direction)), anchor lines (4 (two points) - 2 (each point must lie on the One) = 2 d.o.f .), and anchorcirdes 
(6 - 3 = 3 d.Q.f. (center point coordinates and. radius). 

• A,fui^er characteristic, of the nnodei anchor objects is that their number will typically" be sraaM, but sufficient to 
enable accurate positioning of the geometrical objects. More anchor objects than needed to.derive ttie geometric 
transform may be selected when they are" considered topologically relevant In that case, an over-determined 
solution is derived that fits a gmmetiic transfonnation by minimizing e;g. a least-^qua^ 

• the type of image locations that modei anchor object represent can eitherbe natural orartificial landmark en^es. 
Neural anchor objeote are typical, radiologicai weji- manffested diasfnostic poisitions. x>r reference lines. in the. 
image. Artificial latidniarK entities comprise specifically designed matters introduced in ttie radiogiaphed subject 
or man-made objects sucli as prosthesis shapes. 

Exshiphs of ancfiar object ' ' 

{001.:^ Examples, of suitable anchor point obje<^ in radtogmphic linages, of a^il-faglegexamiriation aree.g. thetip 
of thegreater'toxwhanfe^^ 

|0ai3]. -&;£impl^ of suitable anchor line object i^^ 

• the horizontal line connecting the iliac crest top points. For the iTiaccrest has cit^ 
; its radiographic position is iess affected by pelykj titt. 

• the topmost; bottommost horizontals, tangent to 

• the feftripst and righ^ ; 

• the line connecting both inferior rami of the pubis , , ^ 

• die litoischiai line j(KoliIef^ line). Anatomically the superior ilroischial line is formed by the broad cortic# surface ' 
of the sidatiG notch (which lies posterior to the acetiabular flqoO,. and the inferior ilioisr^ial iine.is formed by the 

.. cortical surface of the anterior ischium (anatomically this litre does not con^espond to the acetabular floor) . 
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• the line connecting left and right teardrop. The true floor of the acetabulum corresponds ^elRe^diographlc tear- 
drop. The teardrop lies in the inferomedial portion of the acetabulum, just above the obturator foramen. The lateral 
and medial lips correspond to the external and Internal walls, respectively. The medial walUs a relatively constant 
radiographic finding and is not significantly distorted by small degrees of rotation, unlike the illolschial line (Kohler's 
line), which may sometimes not consistenly represent the true acetabular floor (on straight AP radiographs, the 
ilioischial line will overlie the acetabular floor, but with an amount of rotation, this relationship may not be met) 

[001 4J Each of these lines is actually a 3D line; which projects as a line in a tw^o-dimensional image. To the extent 
that the liries intersect in three dimensions, the representation as a set of (intersecting) anchor points may be used 
Instead of a set of anchor lines. 

Coordinate frames in model and target image (fig. 1 ) 

[0015] In the following explanation, a number of coordinate systems are used. ' 
Let the model resp. target image coordinate system be defined as (u.i')resp.(x,y), with the xaxis running from top to 
bottom (that is having rows as the running coordinate) and the j«xfs rurtnjng from left to right (having columns as the 
running coordinate), u and v axis resp. x and yaxis fomfi a right-handed coordinate system; with positive angles de- 
parting from the u (resp. x) axis In tJie counterclockwise direction towards the v (resp. y) axis. 
[0016] The anc*ior objects in the model (resp. tai^et) image define a local second coordinate system(t/,i/)resp.(x', 
y), into whfch the objects are encoded relatively. For example, when a set of anchor points is used for the model, the 
directional line segment from the first point of the set towards the second point of the set defines the ^axis. The 
perpettdifcular y^is points positively in a direction sucfi that both axes jointly fonn a right-handed coocdinate system. 
A tfiird coordinate frame((/'.v') resp. (V'.y') may further be attached to the geometric objert in the model resp. target 
image to describe the location of its constituent parts.. For example, a coordinate system may be attached to a ROI to 
describe the pixels' position inside the ROI, in a way similar to the image coordinate system thait describes the pixels' 
position in the original image. The defining parameters of the ROI's coordinate system are the Ideation of Its origin In 
the (t/,i/)resp.(>',y') coordinate system and.the angle of its positive t/'resp.yaxis with respect to the positlvet/resp! 
axis . The direction of the positive V resp-. /axis fs then establfehed so that the o"and v" raspectlvely X" and y"axes 
jointly form a right-handed coordinate system. 

LIST OF FIGURES 

[0017] 

Fig. 1 . Coorditiate systems used in the context of the present itiventlon: model cooitlinate and (b) target coor- 
dinate systems: image coordinate systems u^v resp. jf,y , anchor point andmapped anchor point coordinate sys- 
tems uy resp , x*,/, and ROI and mapped ROI coordinate systems i^sp. x",/. 

Fig. 2. Bilinear mapping of a point P fefenenced in a quadrangle ABCD to a point P referenced in a quadrangle 
A'B'C'D" 

Rg. 3, Baiycentrfc coordinates: (a) afHne combination of two jj&frits, (b) afflne combitiation of three points. 

Fig. 4. Projective mapping of a point P referenced in a quadrangle ABCD to a point P' referenced in a quadeangle 
A'e'C'D'. ■ ' 

Fig. 5. Regions of interest in a hand examination encoded accofdfng to a relative encoding. The ftOI's mapped 
from the riiodel template towards the target Image will be aligried with the actual bone' axes fff the image. The 
anchor points (flg. 5 (a)), taken in pairs, determine the ph^ngeal and metacarpal ROI's (flg. 6 (b)); a quadruple' 
otanchor points (fig. 5 (a)) detenriines the carpal l=JOI's (fig. 5 (c)) In this ©cample. 

f^g. 6; Regions of interest In a full-leg eWinafion.eriooded according to an dMoluteenoo^ ROI'smapped . 
f nbrii the model template towards the terg^ image femain aligned With Ihe.lmage botders. "n>e anchor pointe are ' 
the top of the greater trochanter, the centero in the 

template parie>. Greater trochanter arid center of knee serve to determinetfje femOi^Hneck and femoral shaft as 
welt as the knee joint flors in ttris ex&niple. Center of knee and center of ankle sen/e to, deteriiiine the tibial shaft 
as well as the knee joint and anide Joint ROI's in-this e)^^ 
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Fig. 7. General system overview of mapping geometric objects in images by means of aRehpr-«bjecte. 
DETAILED DESCRIPTION OF THE INVENTION 

[0018] The geometric mappings that are considered in ttie context of the present invention can be classified in either 
linear mapping functions or non-linear mappings such as bilinear mappings, polynomial mappings and thin-plate spline 
mappings. These mappings are detailed along with the computation of them by means of correspondences between 
model anchor objects and image-mapped anchor objects. Special important cases are indicated. Decomposition of 
linear mappings is derived to be used in relative mappings, that use the full established transformation matrix, and 
absolute mappings, that use a reduced transfonnatjon matrix by omitting one or more components that are obtained 
by decomposition of the full transfonnation matrix. Next the encoding of geometric objects in the model anchor coor- 
dinate system Is described. Finally, the result of applying the computed transformation or reduced transformation to 
specific geometric objects is derived. 

1 . Geometric mapping transforms 

1 .1 Linear mapping functions 

[0019] This class of spatial 2D mapping can be expressed in terms of the general linear homogeneous 3x3 trans- 
formation matrix Tand is commonly refen^ed to as a homography. This transfonnation handles scaling, shearing, ro- 
tation, reflection, translation and perspective in 2D; A 4x4 transformation matrix in homogeneous coordinates similariy 
expresses the class of spatial 3D mappings. Without loss of generality, the third dimension is ignored in derivations, 
and the 2D object mappings only-are considered in the sequel (that Is mappings between the image xyand model uv 
coordinate systems). Coordinate, systems are typically right-handed, in that the xaxis is running from top to bottom 
(running inside a column), jfaxis running from left to right (running inside a row); xy tommg a right-handed coordinate 
system. 





a„ a, J a,3 




X 




'u 


2- = 


«2i ^22 «23 




Y 


=r 


V 




>JI «32 




1 




y 



wherein the homogeneous coordinates have been nonnalized to have w=\ . The wcoordinates denote model position; 
the xy coordinates denote image position. The homogeneous notation has the advantage that a translation is modeled 
as a matrix multiplication instead of requiring an additional vector addition, hence it may be encoded and computed 
simultaneously vWth the rotation and scaling components in dedicated hardware. 

[00201 The 3x3 transfomiation matrix.can be^ be understood by partitioning it into four separate sectjons. The 2x2 
st^-mabix . " 

specifies a linear transfomnation for scaling, sheaiing and rotaflon. The 2x1 sub-matn'x 



specifies a translation and the 1x2 sub-matrixlaa^ ^produces a perspective iransformationv The element (agglis 
responsibfe for overall scaling. . . , ^ 

[0021] Linear maffwig functions m 3D are represented by the 4x4 homogeneous matrbc muftiplfeation 
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In a similar way, the 3x4 sub-matrix 



^^21 "222 ^1 
flj, fljj ^34^ 

represents the affine component (rotations,' transiatfons. shearings), the 1x3 sub-matrix (a^^ a^2 a43)represenls the 
projectivities and the coefficient is responsible for overall scaling for homography mappings in 3D. 
[0022] The matrix rspecifies the forward mapping functions Xand Vthat transfomi mocfel source Input images in 
tlie ((/,v) coordinate system onto tair|et output Images in the (x,]/) coordinate system. A similar transfonnation applies 
to the inverse mapping functions U and. I' to map the larget image into the input image. 

Special eases of linear mappings 

Euclidean and Euclidean similarity mapping and similarity reflection 

[0023] Euclidean mappings offer four degrees of freedom, in the case of isotropic scaling, and are able to map a 
bounded line segment into a bounded line segment (with different lengths and orientations). These mappings are also 
temied rigid body transformations in that they leave the shape of the transfomied objects similar up to a scaling factor. 
A rigid body transfomiation consists of a coimposition of a rotation about the origin over an arigle 6 wj-.t. the u-axis in 
the counterclockwise direction, a translation (T„TjJw.r.t. the origin and scale factors S;^St^pplied to the coordinates 
uahdv: 
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Scaling T to have a nonninity term can reduce the S varlatiles In this mapping to 4. 

(0024J Considering isotropic scaling by a factor S, the combined Euclrdean mapping can be written as 

'Scos0 sine 
-sine SCOS0 
.0 0 1, 



[0025] The Euclidean similarity reflecbcn mapping is obtained by concatenafing this matrix with a reflectton matrix. 
For exanple, reflection arotind the {^BXis f6s^. lAdxistu^e represented by theina^ 
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In three dimensions, a rigid body transformation is characterized by six degrees of freedom (three translation compo- 
nents and three rotations around each of the image axes) 



n6t oos0^ 



0 OYl 0 0 

0 0 [o cosfii, -sin^„ 

1 0 jo sin^„ cosff^ 



cos0„ 0 sin^, . 

0 1 0 
-shi0, Q costf. 



1. T„ 
0 1 



{0026] The first matrix represents a TOtatica-in the counterclockwise direction around the tv-axis, the secortd and 
third likewise represent rotation around the u- and i<-axis. respectively. The angles of rotation are the so-called Euler 
angles. The fourth matrix represents three-dimensional translation. 

Affine mapping 

[0027]; The general representation of an affine transformation is 



^3 

0 I 



Division by the homogeneous coordinate \a/\s avoided by selecting w=vf=V. The affine mapping corresponds to an 
orthographic or parallel plane projection from the source plane (u,v^onto the target plane {x,y). 
[0028] Affine mappings in 20 offer six degrees of freedom, and are able to map a triangle (three non-collinear points) 
into a triangle. An input rectangle can be mapped into a patallebgram at the output. Affine mappings result from 
applying non-zero off-diagonal tenms in the scaling matrix. Consideringthecase where a,|=^=1 and%,=0. By making, 
a a^2 ^ Of x>s made lineariy dependent on u and v , while y remains identical to v. This operation results- in shearing a: 
rectangle, with sides parallel to the u and »^axes, alongthe ucoordinate axis into aparanetogram In the (Xij^coordlnate 
system. The side pairthat was originallyparallel to the vaxis is now oblique w.r.t. theyaxis. Asimilarnon-zero coefficient 
a2i*0 restilts In a shearing parallel to the vcoordinatiesaxis. Thepeneratformof an ^iiie mapping may^ therefore 
a(so be wrtiten In decomposed form as . 



- r, = 7i; 0 1 o Lr, 10 

\0 0 lj[o 0 1 j 

[00291 The lnvetse of an affine fTiappingB its^^ihe, and fecG^ 

IPOiiOJ In three dimensions, each new coordinate is a linear combination of aH three, and ffie shearing opsraiiof s 
geometfically pesultin liansforniirig a oibe or rectangu^ 
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Projective mapping r^~-. 

[0031] A projective mapping is able to describe object transformations obtained through perscective projections onto 
the viewing, projection or detector plane. In general the coefficients [a^^ agaJare nonzero, and hence a projective map- 
ping offers eight degrees of freedom. 



'5 Where x=x*/iV and y=y7w'. 

[0032] Projective mappings preserve lines in all orientations. That is, lines map into lines (although not of the same 
orientation). In contrast to affine mappings, projective mappings offer eight degrees of ti'eedom, and are able to map 
a planar quadrilateral into a planar quadrilateral. 

Perspective transformations preserve parallel lines only when they are parallel lo the projection plane. Otherwise lines 
20 converge to a vanishing point. 

[0033] The fonward mapping functions of a projective mapping are obtained by dividing by the homogeneous coor- 
dinate v\t 



^y^_a^u*a^v+a^ 
w'~ a^^u+a^v+a^ 



[0034] The inverse of a projective mapping is itself projective, and is computed from the inverse rl=a<lH,Tp)l<iet(Tp). 
[0035] The projective mapping may by used to calibrate an image such that measurements can be performed met- 
35 rically in the mapped image. The points may be landmarks at a certain plane, possibly rotated around the x and y axis, 
at a certain depth in the patient. The mapping may also be used tobring two planes of two images tatien at two different 
times into metrical correspondence, i.e. distances in the mapped plane may be calculated in an Euclidean w/ay and 
•compared vyithout any further conversion. . ■ . ■ • 

40 ^Computation of the Hnear mapping function. 



[0036}. ..Thematrix TghQldsQnfy4.unknowns andean he wfftteaas 



a b 
-b a f 
0 0 



[00371 ; The coefficients can be detemiined by establishing aconfispondence between two points in th 
and target image. Let (t/^iVj^resp. (Xfc.yj^.lc=0,1 be two points (e,g. endpolnts of a line segment) in the model image 
55 resp. taigetinage. Each pair of corrBspbnding point generates two equations in the unknown coefficients by rewriting 
theequatk)ns-X(M:)resp. yrft/,v,)asfoll(3wsj = . 
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Written in matrix notation. 
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[0038] This system of equations is solved by linear system solvers Icnown in ttie^rior art. The coefficients ma/ now 
be equated to the coefficients of written in terms of the concatenated elementaty geometric operations to determine 
constituent components of translation, scaling and rotation: 



e s= arccos(a/S) 

[0039] In three dimensions, a rigid body transfomiation has 6 degrees of freedom, requires at least two ptrfnt corre- 
spondences to establish it. . 

Mine case 

. [0040] The matrix holds only 6 unknowns 6 (6 degrees of freedom) th^t can b& clete(ito|ined by e$tabllshlng[ a 
con-espondenee between- at least three nisn'-co-iln^ points In th'^Yi^odel i^nigg dnd target ihl^e. Let (Uj^v^resp. ^ 
.2 be three points (e:g.'comer points of a triangle) in the model image re^; target )ma@eV Each pair of corre- 
sponding points generates two equations in the unknown coefficients by rewriting the equations X'(u,v)resp.y((/.v)as 
foHows: 



Wntten ini matrix -notation, 
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This system of equations is solved by linear system solvers known in the prior art 

[0041] In 3D, the affine transformation in homogeneous coordinates has 12 degrees of freedom, hence it requires 
at least 4 non-coplanar point correspondences (or 4 intersecting planes). Hence, these transfoimations in 3D are able 
to map a tetrahedron into a tetrahedron. A similar system of equations is established to determine the 12 coefficients. 
[0042] The coefficients of the affine mapping are now equated to the coefficients of written in terms of the con- 
catenated elementary geometric operations to detennine constituent components of translation, scaling, rotation and 
shearing, it follows immedlEttely that 



[0043] The determination of the scale S, the rotation 6 in counterciacloMse direction and the shears s^,s^ can be 
retrieved from the coefficients a,b,(/,e by first concatenating the individual matrices and then equating each of the 
members of the concatenated matrix to a,b,d,e respoctivefy. The concatenated matrix is 



rs cos e s{s^ cos e-sme^Ti o] _ 

\_Ssm& Sis„ sine + cose) jy, lj~ 



[00441 VVhen the rotation is preceding the shearirigsj anotherfnatrix results. Thfe decomposition of the latter can be 
used to determine the elementary matrices con^ponding to the reverse sequence. The advantage of decomposition 
is that selective components can be cancelled by substituting tfiem by the unity matrix. For example, when the deter- 
minant of the transformation matrix of the 2x2 upper 1^ rnatilx is not unity, or equivalently Uie matrix S in the singular 
value decomposition U-S- V has non-zero off-diagonai elemente, the transformation is not rigid body, which geometnc 
property can be achieved by canceling the shearing componenf. Likewise, as will be apparent in the applicauon of 
mapping ROI's, canceling the rotartjofi component be useful also, e.g. to have the mapped borders of the ROt 
staying parallalto the image borders In the transfbrrried image. The decbinposltian according to the forward sequence 
of sliearing foltovved by rotation is derived in the fpllov^^ 

• [0045} The notaUon can be sin^ified by intotkiuclng th^ vafiisibfes and B ctefined^s S 00s 8 a and S sin e = B, 
yiefcJing. the folfowing non-linear system of equations 
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[0046] The solutions can be found by rewriting the factors s„.s^. S as a function of A and the l<nown coefficients 
d.cf.e, solving for the roots of this function, and straightforwardly deriving the values for s^s^ B for all values of A. 
Using some algebraic manipulation, the factors s^,s^B can be rewritten as 



[0047] Note that ae-ixfcd©t(T^)=£). The quadratic equation in A then becomes 
(/)^-«^)/4^-2£?e4+D(0-6^ =0 

[0046] This equation has only real roots if the discriminant Is positive, which leads to the conditions 

D>0 and (tfi+e^)>D. The two roots are easily computed by the standard formula for roots of quadratic equations F 

each VEilue of A , the factors s„,s^ B are obtained from their functions of A given above. 

100491 the equations may equally be iBwritten vi^th s„ as the unknown , leading to the solution of a quadratic eqtiatioi 
which has a doubfe root 



[0Q5pj tfi^ same condtOon (/^ 

[0051] Obviously, the original matrix can be factorized In two ways. To reduce computations when the transfomiafi 
is carried oiit by as sequence of coo^ecuflve sh^sMlngs, that solution rriay be sete^ed. which has zero coeff ici eiii 
fofonepfliejsheailngs i^^^^ ho 
mifch the totafofafl indvichralx»rnp.6nCTts of rotation an^^ this matrix repres e i ^ 

the IdenSty trarisfoiTij, impiyiag.no phfingepf fljeniodel shape, aR;art,frorn a translation. This is a(?hieved by subtracai 
the iderilityrmatrbc of each irulividiaicompoii'entof a decomposition (i.e. calculating TR-fwith T^the rotational cornpc 
nent for example), summing the absolute values of ;ail elements of this difference matrix to yield an individual c ; 1 
nenfs deviation measure, finally sumniing all de^^on from unity measures of all components of a decomposi c 
selecting that decomposition which has the least«verall devetion from unity measure. This novel strategy can u- i t 
be complemented by removing small components^ in fh6 sense of deyiatipr> from the identity matrix just def ined, t 
substituting them by the identity matnx, to achieve a.simpler <^ain of elementary ttansfomtations, without saorificir 
mapping accuracy of the resulting combined transfbmidtion. 

[0052] In SO, an affinity has 12 degrees of freedom, hecxfe solution (»h'be derived from a minimum cf <■ c-o: 
Goplanar points, or equivalently using their dual of planes. t)y 4 ndn-eoplaMar planes (Le. fonning a tetrahedrc n) 
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Projective case — -• ^ 

[0053] The matrix Vcan be normalized so that 833= 1 , so that eight coefficients need be detennineci to Infer a projective 
mapping from image data. These coefficients can be determined by establishing a correspondence between four points 
in the model image and target image. Let (o,^ v^)resp. {Xi^yk), k=0...3 be four points (e.g. comer points of a quadrangle) 
in the model image resp. target image. Each pair of corresponding points generates two equations in the unknown 
coefficients agby rewriting the equations X(u,vjiresp. Y(u,v) as follows; 



{0054J Written in matrix notation, 







I 0 


a 


0 










^0 




V, 


0 


0 


0 






^12 
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0 






a,3 






«3 


V3 
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-u,x, 










0 


0 0 




^0 


1 






°n 




Jo 
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0 0 


"1 
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-"lyi 








Ji 


0 


0 0 


"2 




1 


-"2^2 








J2 


0 


0 0 






1 













this is a linear system of the forni Ajfefc, that is solved by linear system solvers known in the prior art. These techniques 
rely for example on inverting the.matrix A, so that the solution Is found as x=A-^b. This yields a solution to the general 
planar quadrilateral-to-quadriiateral mapping problem. Computational speedups are possible in the special cases of a 
quadrilateral to (unit) square mapping and the (unit) square to quadrilateraJ mapping. The general case of quadrilateraf- 
to-quadrilateral mapping can afso be speeded up by concatenating these special cases in cascade. 
[0055] As points and lines are dual to each other ih projective space, 4 non-collinear lines in 2D can alternatively 
determine the homography. Similarly, in 3D a homography hais 15 degrees of freedom (the overall scaling can be set 
to unity without loss of generality), honce if is fulfy determined by a minimum of 5 points, or alternatively by 5 planes, 
'[00S6J A projective transformation cannot be factored into elementafy shearings along thb d6r axis only, (nsfeacfi 
deoonipesftion in a purely projective component and an affine cbrnporient is considered. The purely projective corti- 
pofterit ls:6ble to transform converging lines that are origitialVpai^lel in the model but convei^ge in the Irnage towaWs 
ihs vaErHsMngpoint fbr-tii^ liflddirecfion/The affine comfiofrdrii maamm ior gibtial aiighr^ent and reduc^ondf distdrtiorr 
al6»^«iep^e&j This decoinposifroft'fe dcc^ 

{QOST} befln6. T=T^7>wl»re rp\fe.» purely proj affine transfoirh Ofeflh^ the projective 

compdhferifais- • ■ • 



1 0 0 
0 1 0 



.{0058] Hetice, tfie ^ne component derivect as 
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= tt;'-- 



a~cg b-ch 
d-fg e-Jh 
0 0 



This affine matrix may further be decomposed into a rigid body transfomiation (scale, rotation and translation) and 
elementary shearing transformations as specified previously. As with the affine factorization , components that are small 
may be ignored without sacrificing positional accuracy of the mapped objects. 

[0059] The foregoing detailed description of the linear mappings is needed to describe the application of a fast map- 
ping of complete ROI's or image areas in the sequel. ' ' 

Computation of the linear mapping functions in the over-determined case 

[0060] When the model contains more anchor paints than needed to specify a geometric transformation, then m>n, 
i.e. there are more eqtJations than unknowns in the linear system Ax = to. For example, more than two model points 
may be selected to infer an Euclidean similarity transform. TTiis problem is called over-detennnined, and generally no 
^satisfies Ax= t exactly. The underdetermined problem, where m <n occurs less frequently and is generally unwanted 
because it would leave too much degrees of freedom. Three standard ways in the prior art are used to solve the least 
squares problem, all of them are based on matrix factorizations: the normal equations, the'QR decomposition and the 
singular value decomposition (SVD). 

1.2 Non-linear mapping functions 

t.a.1 Bilfn?iaftnapping 

-[0061] A bilinear mapping is the tensor product of two affine functions of one variable each (coordinate variables u 
and V). Geometrically (fig. 2), it means that an artiitrary point inside the unit model square- ABCD is obtained by per- 
forming two linear inteipofations. The one is obtained by interpolating on a pair of juxtaposed edges AD and BC.using 
the same coordinate value uto divide the edge yielding points P and Q on AD and BQ respectively. The other is obtained 
by interpolating the desired point on tite line PQ using the other coordinate value v. 
The tensor product is written as 

rg(u.v) = (X(u.v).y{«, v))=(re^(i/.v),rg^<u. V)). with Ts(,u,V)J^{^u.yf) 

of.thefomi 

T'g (t/,v)'=(m(/-fn)(pi/^Kr)=stfv<f6(»cwd 



(f/,iO={m'i/+n')(p'w-q[')=euv+ft/+gi'+fi 

[0062] The technique whereby each coordinate transformation TB^u,\fiJgjfi,yi\ are bllniear functions is called a 
bilinear transfomration, and the technique of using a bilinear transfomiation for the four-point mapping is called bilinear 
interpolation. 

Computation of the laliinear mapping -;< . • 

AigiebrsiadetetTxnnaUon of a point mapping ■ , . 

|006S| The 8 parameters a...d,a'...d are determioedby specifying the con-espondences between T^(^0)-ii>*<>b yc) 
7"B{0.1) = 6(x,.yi), re(1.1)=C(W2>. 7e(0.0}=0(X3,y3). or equivalent^ 
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which leads to a system of 8 equations in the unlcnowns a...h, or equivaJently, two systems of 4 efquatlans in'the 
unl<nowns a... £f and e...h respectively. 

^0641 The coordinate vatues (u, may also describe the position of a point object Inside a non-square model quad- 
rangle. This point is mapped to its corresponding position in a non-square target quadrangle, following the method 
outlined in the sequel. The transform goes between an intermediate unit square yielding intermediate bilinear coordi- 
nates of. the point in the orig|naJ quadrangle. Denoting with U.Whe Cartesian coordinates of the comers of an input 
quadrangle., and denoting with f,J and a; V the Intermediate unit square coordinates ordered in clockwise direction 
starting from the origin and the output quadrangte Carteslan corner coordinates . 



H/o /f^yHOOl t}' 



J={/o/i/2fer=[0110}.' 



Let (u.n) denote the Cartesian image coordinates of an input <mDdel) point. -let i(/j^deti6ts the bilinear cddWInates of 
the input point w.r.t. the input quadrangle, which are also the Cartesian coordinates: In the intermediate unit square 
coordinate systcni, and Jet (jf,y)denote the output Cartsislan coordinates of themapped input point now referenced w. 
r.t the output quadrangle. The forwarti transfomi matrix that mi^s the unit squate comer points Into the terget quad- 
rangle's vertices is denoted bjf 
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lo/o h Jo I 

hJl h Jl 1 



[0065] The mapping is then determined following four steps (fig. 2): 

1 . Compute the forward bilinear transform from the unit square towards the input quadrangle. The coefficients a... 

■ d and e..Jt are obtained by ' 

2. Determine the inverse transform from the input quadrangle towards the intermediate unit square, enabling the 
computation of the bilinear coordinates of an input point referenced against the input quadrangle. This is performed 
by solving a quadratic equation for the value of / 

, with . - 

c^=-ec+gra 

c,=a{/7-v0+'e(t/-«/)-fc+gij 

This equation may have only one root when c^b. Ottieiwfee two roots are obtained as . _ 



■ ' Ond rodt Is retained" to' yieW al5olwi>n^i^.(/^<^, Ifie (^i^ thi^ tteg jhsicJ^ tt^ unif squan^ when the input .. 
.<mod^l).poiiit (u,^lies insWetoein^ut (niocfal) qWdfsngloJTTi'e sototio^ ac* existwhe'afhe discciiiiinant c.-.". 

.: ■ iran|lbhifed fees «it^ Uq, constant (vetiicat lines). lihes^ Jlnes ftrfd oyer,, c^^ the exfetenpe of two so- , 

lufiofta. • ■ . • 

' Asimil^dewafipn of t^ 

res(ion f4f .Whksh hd sotM>ri,exfe^^ ffian.iero) fejra^^ the same parabola, and; 

.aigaih'tvvo Boftffloi^^^^ 
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3. Compute the forward bilinear transform from tfie unit square towards the output quatti-aKglerThe coefficients 
a...rfand e...h of this mapping are obtained by 



4. The bilinear coordinates (/,/)of the input point (u,i/)are now mapped towards their Cartesian position (y,j/)with 
respect to the output quadrangle by applying the forward transform 



a' 
d' 



y = \}J i J 1] 



Alternatively, the position of the mapped point in the target image may be obtained by emulating the geometric 
construction according to fig. 2 whereby the bilinear coordinate / divides a pair of juxtaposed quadrangle borders 
associated with the x axis to yield points X, and X2 , and the bHinear coordinate y divides a pair of juxtaposed 
quadrangle borders associated with the yaxis to yield points V, and Y^. The mapped point is subsequently found 
as the intersection point between line ^and line Vg. 

[0066] Point sets referenced w.r.t. the input quadrangle may be mapped to their associated point set in the output 
quadrangle by applying the transfonns of the foregoing procedure to each point of the set. Other objects relying on 
points in their definition may be mapped lilcewise. Horizontal and vertical lines of the (u,v)plane are transformed into 
lines; however, other lines of the (u,i}plane are trartsfomied into curves of second degree (parabolas). A uniform 
subdivision of the edges of the Inputunit square or input quadrangle is mapped onto a uniform subdivision of the edges 
of the ou^jut quadfilaterat. 

Geometric detemtlnaffan of a pomt mapping 

. [00671 Ttid first component consists in transposing the position of the point in the input quadrangle into a point in the 
intermediate unit square (ofwhich the Cartesian coordinates are equal to the bilinear coordinates of the point) This 

. polfk can be found as the intersection of two Ibids of constant bilinear coordinate /and/ eatii line being tangent to the 
parabola tleffnealing the region in the u.i^ plane for which no Inverse exists. The parabola Is given by the equation c^- 
4£^<^ <0, which is an equation In the variables ii.vaf the input model coordinate system, and the coefTfcients are soidy 
d^endenton the coordinates of the comer points of the model quadrilateral. For there are two solutions for each 
blGneaf 0oo(dinate{|iE>ursotutions forthe (/j) coordinates in total), that solution must be selected that has /jicoordinates 

■befewe^Oan'tflwfieintb^^ 

The'sec6hdcfflrhpo^h6nt^ the position j6f an aibftnary point wfm bifinear <x)ord\nat6s (i,/)^ of the 

■unit square In the terg^fijtfaci^ Thistriansposltion can be.foundtjy f^o jmeariiitefpolati^^ illustrated In fig 
2, first tire /cborcffn'afe f^^^^^^ on tfie edges A'D' andB'C. obtainrng points X^)^ respectively. Next the / coorr 

dinate Is Interpolated on the line ATiXa, obtaining point P'. AltemsMvely, the/coordinate maybe interpolated on the tine 
A'& ami D'C, obtaining poffits ft, ?espeGdyejy, The ift(6rs€Etiofi of iih^'x^zandV^V^aiso yields poirit P'. 
[OOeq • -iTie advanjag^of^^ 
.rhayrepriSs^^t^iofntey^iatMira 
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to-quadrilateral projective mapping assumes the quadrangles' comer points ail lie in the sarrte-pliysieal-piane. 
1.2.2. Polynomial mapping _ 
[00G9] A polynomial mapping is defined by a polynomial function of the form 



The u, I' variables are coordinates in the undistorted model, x,yare coordinates in ttie target image. A/is the polynomial 
order, tfie a,^i),yare mapping coefficients that characterize the transformation. When A/=1, the mapping reduces to the 
previously described bilinear transformation, for which four anchor-point correspondences need be established: 

[0070J In general, the minimum number of anchor points needed is (Af+1)2 fof a polynomial mapping of order M. 
When more anchor points are available, a pseudonnverse solution for example retums coefflcients thati)^ approx- 
imate the tnie mapping function. .r^:^ 
30 ^[0071] An alternative expression fbrpolynomial mapping ofdegree N, where degree is themaxImuEn sumdf powers ; 
of uand vcoordinates, is 



When N=1 , the mapping reduces to the previously described affrne transfdrm^on^ for which three anchor-pbinieor- 



[0072] When the anchorobjects cover the ftill iniage area, Ihe polynorriial transfofniations giy^ above are low-order 
global tn^fing functions dperating over t^^ 

"than the the spatial CQn:espojidence.pf the mapped obje^'wtth respe<$to.ttie imaged sftTUCtures wH! 

: bebelterlhOTt^^^ -. : .. ...... - . ' 

(0073] Gni^^ the.cQtnputed m hovveveFgen6rs%^a flpa^^^ 

pointposttioh i|i the taiget Imaige. Uh^'ofq'en^^afe mapped b po^q^^^ in genefsil. 'RtereforEt, to retain the 

.}i^^Sk\^^t^:^ irte^ taigef (rnagg. Hrtes trKQ? b6.<jdfin©^ oaJhe"^is of points, vAfeti ^re tnapped 
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according to the mapping functions X[u,v>), Y{u,\i), and a line object must subsequently be recl«iwfT*rthe'larget image 
based on the mapped points. 

1.2.3. Thin plate spline mapping 

(0074] When the number of anchor points is larger than needed to compute the transformation , an over-detennined 
system of equation results. The solution coefficients are such that the mapped anchor points will not exactly coincide 
with their corresponding anchor points in the target image, but will fit them in the least squares sense for example. 
[0075] An exact interpolation of the target anchor points can he achieved however by adding a corrective non-linear 
temi to the linear affine mapping functions; 



y=b^+b,u-^b^v+Y,CjU(r.y 
/■-» 

The first tenns represent the necessary affine portion of the mapping, as introduced and computed previously from the 
Af anchor point correspondences, the lasi term represents the radial basis function log fi with /^=(i/-o^^2-f(i/- 

ij)2. The A/ anchor points (Up i^jare called the control points and may be placed in an arbitrary (non-rectangilar) manner 
overthe image. 

■2, Expression and location of geometrioal objects in the anchor bbjeef model 

[0076]: The model anchor objects are used to define acoordinate system in which the geometrical objects are ref- 
etBHced. In principle any model objects may sen/e this purpose, but typically objects that are easy te map will be used. 
In the context of the present invention, an anchor point set or an anchor line set are suitable objects! 

Euclidean coordinates and Euclidean object mappings based on a line segment 

[0077] Two points a and 6 of a point set containing at least two points may be chosen to attach a tocal model coordinate 
system to. The xaxis may be chosen to coincide with the line connecting both points. The sense may be chosen from 
the first point towards the second, and the length of the segment ab may be nomialized to unily. The direction of the 
yaxis may he chosen to be perpendlcuiarto tfre xaxis. and the sense of the yaxis is such that a right-handed coordinate 
system Is formed with the x-axis (rotation from positive xaxIs to positive y axis in the countercloclairise direction). In 
this way an orthononnat coordinate system is fomiect, which is suitable for use in Euclidean object mappings. 

Baryiiehtrie coordinates and affine object mappings based on a triangle 

f007»J An aWne combination of two points and is defined as P= a^P^^P^ with at+a2=1. This point P lies 
somewhere on the line segment joining P^ and P^VOstsX. An equivalent fomi. P=(1 -iiP^+tP^P^^l{Pz-P-i) is an affine 
transfomialioii with = Mf any a/is less Vnati zero or greater than one, then the point wi« lie outside the line segment. 
If eWjero^ or Ota fe zero, then the point Witt coindde with either P2 or respecHveiy ^(^^^ 

P in a plane can he defined by as a combination of three points -P,,P2.P3 defined as Pt=aiP,-Hi2P2+a3p3 with 
ai-«3!2«^1. This affine combination defmes a point in the triangle AP^PaPg as P=P,+«2(P2-P,)+^(P3-Pi ) if any 
is lesis than zetio or greater than one, then the point wiB lie outside the triangle. If any a/is zero,4hen the point will lie; 
on We ixJOndacy of the fti^gfe (fig.'3 (b)>. 

A coordirialo frame for affine /h-^)ace or shorflyfraime. wh (n=23r, 
is new formed by choosing an origin point O and two li^nearly Wepgncfent.vectors. Jn two dimensioiis this irarne is 
denoted (r,.V2.i^. Any point Pfcan be Written uniquely ^P=p^^:fp^^ \^we define points Po-Pi.^s t'V A, = O. 
p1==^d^:V■^,P2=C^■.V2 and 6e\m ft toije p^lKpi+jj^), then p can be e^tiivali^ be written as P^qPo+P- ^ PoP^' 
wihfefeby jp^+Pi+^l. In thfe form, the V^ues (ft;pv.4)ar6isailt^ the bsx^iiibncaQOti&mka^ to f i ps^nts 

('*o.'\;i^2l. a><* are used td descfifefecbjkte In an^neitsjohlMatefram^ il^iaorta^ a iriangte 
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^PoP^ P2 The vectors (k, , v^) obviously represent the sides P, - P,, = v■^ and Pg - Po = ^2 re«pective)y-of the triangle 

Two frames (!', . ^2.0) a"d (v'l , P'a.O'jfomied by two triples of points can be related to eacli otIietUhat is ttie coordinates 
of a point object in one frame can be converted to coordinates of a mapped point in ttie otiier frame by expressing tt\e 
5 vectors and origin of the first frame in terms of the basis vectors and origin of the second frame since the latter form 
a basis for the space of vectors. Thus the change of coordinates is accomplished viaa matrix multiplication. The rows 
of the matrix consist of the coordinates of the vectors of the old (first) frame F relative to the new (second) frame F, 
and is denotes as follows 













^2 


0" 


V,' 










PlV,+P2^2+0=[Pi Pi 1] 




=[p, Pi 1] 
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-[p. 


Pz 




15 
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O' 






0' 



and first two tenns of the product 



[Pi P2 Ij 



^2 U 



represent the new coordinates in the frame (i/i',V2,a>i 

[0079] For any of the vectors of the first frame, their coordinates in the second frame, and Hence their, associated 
row in the matrix E are computed by projecting the vector of the first frame on edch basis vector of the second frame. 
LetTrepresentageneral vector in the targetframe(^1,^'2■0')- If we define 



then p',,p'2 represents thecoord!f*at€»ofFln(i?'i,v*2,^rBy tetttf»aT=5f|,l>e ^aHlitfes ^'f>'2r4prteerii.e|^^^^^^ 
lilcewise by letting 7=^2' quantities p',, p'g represent and e^. By letting ftsff-O, ©3. and &2 are oarculatecl; ' 
[0080] In three dimensions, the niatHx Ethaf rotates a ft^UTie-(v,, 1^2,^3,0) with a frWie (v^^.V'a.v^^.p'jjJs 4x4 raatn% ; 
and is computed using Crame;r'sru|e by defifling the deteiminants ' ' . ' ' , 
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and calculating p'j=^,p'2=-2-,p'g=^, which represent the coordinates of Tin (v'^.v'^.v'g.U). By letting 7=v^ ^2.1^=^3, 
1=0-0, the four rows of E are confiput^. 

[0081 J Obviously, the first frame represent the coordinate system formed by the anchor point In (he model image, 
the second frame represents the coordinate system formecf by the mapped anchor points in the actual image, and the 
conversion describes the way objects defined in the model coordinate frame map to their corresponding objects in the 
actual image. 

[00&2] This affine mapping method can be extended to include vectors as well. Let U be a vedor In a first affine 
frame, and let PSnd Q be any two points such that v=P-Q. We define the mapped vector by an affine fransfomiation 
F into the second frame to be the vector given by F[v) = F{P - 0) = F[P) - f^Q. 

[0083] A line can be defined as the combination of a point and a vector, L=P-hxv, and the mapping of a line object 
Is determined by F(fg = /=(P}-Kx/=^P)r. 

Projective coordinates and object mappings based on a quadrangle 

[0084] Similarly as a point may be uniquely expressed with barycentric coordinates in an affine frame based on a 
triangle, a point may be uniquely expressed in coordinates defined by a projective frame based on a quadrangle. A 
geometric and an algebraic procedure to map a point referenced with respect to a projective frame in the model image 
towards Its corresponding points in the projective frame of the target image are outlined in the sequel. 

Geometric procedure 

[008S} Similar to the preservation of ratio ol distances and the rule of three in affine or bilinear coordinate mappings, 
the projective mapping preserves the cross ratio of distances between four collinear points. The procedure further 
relies on the fact that the intersection point of parallel lines, maps under a projective transform to the vanishing poinf , 
forthat.llne.directfpn. . . . . 

[008€| The ctDSS ratio e/(Pf/fe;ft j3<pof four collinear poin^ Pi ,^,/33,/tj^ In 



In which is the Euclidean !ilistance.betv»66n points P/ and fjwith affine coordinates {>f/:y,)resp. (xj,y}. The Euclidean. 
..distance A^^is computed framthe homogeneous coordinates (X,. V^M'/jand [Xj, VJi M^of the projective points p, andpyas 
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[0087] If one of the points lies at infinity, the terms in C/tPt ,P2;P3,P4)cancel and the cross p5l5"feduces to the ratio 
of the remaining three points. 

[0088] Referring to fig. 4, the projective / coordinate in the unit square reference frame is cQtTputed from the affine 
collinear points U,,P,l/2>'^''i the model quadrangle as 



The projective / coordinate in the unit square reference frame is computed from the affine collinear paints l/g, ,P, VP2 

in a similar way. 

[0089] The mapped point P is now obtained as the.intersection of line X< VP^' (or equivalently XiX^.and line VPj' 
(or equivalently Y2):The point (and similarly V,), that is the mapping of point V2, is constructed by drawing it on 
line D'C afa distance from point D towards Csuch that the cross ratio of the resulting configuration Ci{D ,Y2,C .VP^i' 
in the target image is equal to the cross ratio Cr[ .PjUj, VP.,) in the model, which cross ratio is equivalent to the cross 
ratio 1/(1 -;) based on the projective coordinate /. The point X-, and are similar^ constructed using the. cross ratio 
based on the projective coordinate /. 

Algebraic procedure 

[jOOSOl The algebraic procedure to determine the projective mapping of the model point referenced with respect to 
a model quadrangle is based on the projecUye transfonnatiGn Testablished from the correspondences of the foui" 
quadrangle corner points as outlined.previously. The homogeneous coordinates of the position of the mapped input 
point in thetaiTgetimage are straightforwardly obtained by multiplyfng the projective coordinates of P with T. The image 
coordinates of P'are subsequently obtained by dividing by the hornogeneous coordinate, that is • ■ ' 

"83,0+3321^+833 

3. Mapping of geometrical objects 

3J. Application of the mappmg Ivnctiori to the geori\etd<^ objects' defining paramet 

(009i j Object? that are defined iri the local coordinate system are now mapped using the cpmputed trahsfcrm into 
the output image. In the sequeL the mapping result is described for. some elementary, frequ.entty .used, geometrio, i. 
o|)j»^. The linear pfoiectjve mapping is of patlicular interest, because of ^ts invattenoepropetti^Wfr-t the shape of - j 
the mapped object .. . . 

Pqiintiri^pplng ; . -.^ , ., ■; ;. . -■. '■ ■ 

1^92] . Roin t?t. are. f undamerjtal geptnetnc Qb|eds-bgcajuse ttiey can be used in a direct, waiy to desaibe : anatom icat 
focations,^OF in an indirect^y^^ spad^y W9;K^.ciifBWisiQfial objeots such as.lines andoirdeSi Given a transform T 
sp^piffed by .estatjfishing a cofrg spondence between andior objects in inodet and taiget image as outtiried before, a, 
modef ppint.x^=(^i^Ja rnapped h^^^ 

Line mapping 

:{0a93J IJhes rern^ Ones V«*5ler an Euclidean jCsiiriila^^^ anda projecave transfonrt. Under a. 

^biJine^.faOTftifomj/hoyrever, pnes regiahi oiily others ise: ,; 

.ttiejjr'.^ttahsibfnvsdinte^ 
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Rectangle, Polygons and Region of Interest (ROI) mapping and ROI extracting - ,J"T.-^ — 

[0094] A polygon Is made up of a set of plecewlse linear borders, hence it remains a polygon under an Euclidean 
(similarity) transform, an affine and a projective transfomi. Of special interest is a rectangle for it may represent a region 
5 of interestin the Image. Rectangles remain only rectangular underan Euclidean similarity transform An affine transform 
will deform a rectangle into a parallelogram, since parallelism is preserved. A projective transform deforms a rectangle 
generally into a quadrangle. In tfie sequel a method is devised to map rectangles underan Euclidean, affine or projective 
mapping that presen/es the target shape of a rectangle. 

[0095] A rectangular ROI may be encoded in a coordinate system (i/.i/) resp. (x'y) defined by the first two model 
10 anchor points as depicted in fig. 1 . Two types of encoding are discerned, and are termed relative or absolute in the 
context of the present invention. 

[0096] Relative encoding is the term introduced to denote that two points, defining the base of the rectangular ROI, 
together with the height of the ROI are specified to detemirnethe ROI. ROI's parallel ta the image borders in the {u,i/j 
model coordinate system wiir generally map to ROI's non-parallel to the image borders.in the (x,y)target coordinate' 
• »5 system. Hence, due to the rotation component, the borders of the resulting mapped ROt will not stay paraHlel to the 
origlnat model ROI's borders. 

[0097] This kind of relative mapping is advantageous for e.g. bone age scoring, where the rotation of the fingers 
must be taken into account when extracting a rectangular ROI, such that the bone axis in the resulting extracted ROI 
is parallel to the ROI's image border. 
20 [0098] Absolute encoding is the tenri introduced to denote that only one base point, defining the origin of the rec- 
tangular ROI, together with the width and height of the ROI are specified to determine the ROI . The rotation component 
is left out in this kind of ROI mapping. ROIs parallel to the image borders in the(t(,i')model coordinate system will map 
to ROIs also parallel to the innage borters in the (>f,y)target coordinate system. Hence, due to neglecting the rotation 
component, the borders of the resulting mapped ROI will always stay parallel to the original model ROIs' borders. 
2ff [0099] This kind of absolute mapping is advantageous for extracting ROI's at full resolution in e.g. a full teg exami- 
nation to allow for better positional accuracy dunng placing the measurement points. These ROfs typically comprise 
regions along the femoral and tibial shaft, and regions in the femoral neck area, knee joint area, and ankle joint area. 
lrrespe<Aive of the actual bone axis angle, the user typically wishes to extract a ROI that stays parallel to the target 
image bordets. Hence, the rotation that is Implied from thetransfomiation matrix must be ignored. This can be achieved 
30 by only mapping the ROI's origin by applying the transformation matrix to the ROI's origin in the model, and further 
leaving the angle of the model ROI fixed. The drawing of the rectangle is then completed in one of two ways. Width 
and height of the mapping rectangle are detemiined from the scale factor of the decomposed transfonnation matrix, 
and the rectangle Is drawn such that parallellness to the model rectangle is retained. An alternative is to map the 
diagonally juxtaposed ROI comer point by applying the transformation to it and further complete the ROI drawing. 
35 respecting equal angles (i.e. parallellness) of the corresponding model and target ROf sides. 

[0.109] Polygonal objects may be mapped in similar congruent way, either relatively or absolutely. Relative mapping 
is achieved by mapping a base lino arid-constructing the remainder of the sides, and further appfying the scale factor 
to the length of the sides andrespecting the inter-side angles of thepolygon. Absolute mapping Is achieved by mapping 
the polygon's origin point, and redraw the polygon applying the scale factor to the sides' lengths but leaving the sides 
40 pamile* with the corifespondln& model polygon sides. 

(OlOtf Tfie encoding of a rectangular drpofygpnaf ROI in the modelimage proceeds as follows (fig, 1 ). The u* com- 
ponent of the{i/, V) coordinates of the base points of the rectangular or polygonal ROI are detemiined by calculating 
the speed with respect to the anchor line foniiedt>y anchor points and The speed of a point with respect to line 
Si % is oquai to the relive prt^ected dis^nce from the origin of tiie coordinate system along tne if iak . T~he 
. 4s origin of tiie ((/,vO coorcfinates .coincides witit the anchor point a^. A prelected point coinciding with has speed d 
■ (sucli'pBintfiespfttfTe*«^s>. AproJeot&dpoirrf6oiiiddingwlA^ghass^^ 

32 hai s^ed btetween 0 and f , ariypoint pn^ecting before % hasipeedleSs than &; any point projecting after a^ has 
speed gi^aferlhan 1 . The iength between and is tfius nomied to unity. Tfils lerigth also nonns ttie v'coofdiriats. 
The 1^ axis is <»nstnJcted perpendicular onto the axis and fias a positi^^ 
: .60 coordinate system. Tiie V coordinate of a point js detemiined by projecting the point onto the v" axis and dividing the 
projected distamw to tlie origin V^^^ 
. ^102). In the relatiyeenicoding, bibtli base points dfredtengular region are encoded in tills Way, along with the height 
of tftem5fauiglo.'ln tife abs6ltit6eri6od^ 

[Ot(»} • f=ron1 asmall number efaindhoppofiit«<rrespon«^^^ is comput^i, and applied 

» to the( tjase point (s) of tliie model reetangle.^ 

[0104} Id tiie relative casei the scale factor that is E^plied to 

the distance between th<3 mappf^d re<^gle's basepoints to the d^tance between the model, rectaiigle's base points. 
{0105} in tiie absdutec^,1he scale factorthat is applied to tWdtti and iieight Is 
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the affine or projective mapping matrix into its elementary geometric operations of scaling, tranejatiofirfotation, sharing 
and perspective components as derived previously, and secondly taking the scale factor from the scaling matrix. This 
absolute mapping ignores the rotation component in the geometric transfonnation. by concatenating only scaling, trans- 
lation, sharing (for the affine mapping) and perspective (for the projective mapping) components. 
[01061 The absolute angle ROl mapping and ROI extracting has the obvious advantage that the ROI stays aligned 
with the target image coordinate axes x and y, such that the spatial interpolation to extract the ROI may operate in a 
separable way along rows and columns. 

Circle and ellipse mapping 

[0107] A circle remains a circle under an Euclidean (similarity) transform. The circle's center maps as a point as 
described before, the radius is scaled with a scale factor S as determined before. 

(01 08] Under an projective or affine transfomi, a circle isprojected into a conic or a separate kind of conic, the ellipse^ 
hyperlDola and parabola object, the parameters of which can be determined from the transform coefficients as describecj ;' 
in the sequel. 

[0109] The general equation of a circle in the model is given by 

(u-Uo)^(v-i/o)^=r^ 

[OttO] Writing the projective transform Tthat maps (u,i)model points into (x,y) target points as 



X 






y 




[; : 


1 







wherBin the coefficient /may be normalized to 1 since the transfomi is only determined up to a scale factor. The equation 
that relates (u.vjto (x,j^is then given by applying the Inverse transfomi P^, which yields .- .Vr: .- 



ei-fli 
fg-di 

dh-eg bg-ah ce-ArfJIl 



ch-bi bf -celvx 
ai-cg cd—afVly 



For tbe pure affine case for which g=h=0, the (i/.i/)to '(jr.y)relationship simplifies into 



- e hf-ceYx' 
—d a cd-cfViy 
0 0 ae-Mj[l 



'A" B" C'*Jx' 
JD- E" F" 7 
G" H" /"J[l_ 



Introducing this projective resp. affine equation into the equation of the circle yields a quadnc equation of the mapped 
object m the target «nage, which IS the general equation of a conic in two dimensions 



Or written in matrix form 



[x y i]Q 



Ith Q = 



~ABD 
B C E 
D E F 



in which the coefficients A...Fw& a function of the original coefficients a...h of tite transformation matrix T, established 

by correspondence. 

The conic is a real ellipse object when clet(0) < 0 and AC-E^ >G. The ellipse's major axis mal<e& an angle a with the ^ 
axis given by 



c object under an Euclidean, affine o 



[0111] In a similar way as a circle object, an ellipse object maps tc 
projective mapping. 

Bezfer and Spline object mapping 

[0112] These objects are mapped in the image by first trgnsfomning their control points and secondly regenerating 
the curve (or surface In 3D) according to the object's new control points. Bezler and spline curves remain invariant 
under affine mapping, and rational Bezierand rational spline curves remain invariant under projective mapping, tiiat 
is, tiie transformed curve can be obtained by regenerating the curve soleiy on the basis of the transformed control 
points. In the sequel the definition of each curve is given along with Its mapped form. 



[0113] More specifically, an nth-degree Bezler curve is defined as 



C(w)=2]£!.„(«).^ 0£k<1 

1=0 

■<o [ail 4] The basis blending functions are the nth-degree Bernstein polynomials given by 

/« .[0115] The .geometrlGCQefficlents{P/} in this fomitila are called the control points For /?/=(^/.y/). for 

space curves P/=CAj,y^2^. Denofliig|iv*i.e setfP/J.tlje mappedcort points obtained by applying the geometric fnap-. 
ping to the ofigihat set {PJ, the riiapped nlh-d^tee BSziercurire C(t4 is generated on the basis of the mapped controi 
points, i.e. ■ '• • ■ . • 

SQ ■ ■■ 

; ■ Rattoht^ Be;^ curve , ;= .~ 

..j ' fOlt^ An rth-degreie rational B§ziercu^ 



EP i 598 778 A1 



C(u) = J^ _ =Xi?,>(«)^ 0<«<1 



represent the rational basis functions for tliis cur^e form. Similar to the mapping of polynomial Bezier curves, the 
mapped rational Bezier curve is obtained through a geometric transfomiation of the control points, i.e. 



[0117] the underlying ideaforuslngratlonatcurvfif is thatarationatcutveln/>dimenai6na^ 

as a polynomial non-rational Bezier curve in ^n-fl>^imensionai space by introducing homogeneous coordinates, the 
homogeneous coordinate W(u), by which the other coordinates X(u),yit/} <for 2D curves) or X(u),Y{u),Z^d) (for 3D 
curves) are divided, being 



(0118] This operation of division by the homogeneous coordinate geometricaily corresponds to a perspective map 
on the hyper-plane W=^ with the center at the origin. In X-ray imaging the origin physically represents the X-ray point 
source, the hyper-plane Hfcl represents the detector plane. This operation is advantageous in that some important 
curves such as circles, ellipses, hyperbolas, cylinders, cones, spheres cannot be represented by poiynomials. but caft 
be r^resented using the rational functions, which are the catla pf po^omials. 

In the context of radiographic and medical imaging, thfs'operaUon may be used to. derive an analytic representation 
for ttie projection of a 30 space curire in the patient ontQ the detector or irnaging planel Another example is in the field 
of dt^l radiograph reconstruction (ORR) from a CT itriage, wtierb dib^^^e of a curvilinear object in the CT image^ ■ 
posdity fitted to 3D data points, can be represcintBd as radonal Bizjer curve, and its proJeGtion in the ORFI obtained: 
by homogenizing the 3D cuive. To apply the projec^on cfireetly, the cutvljinear object must be represented in the.co- 
ordinate system with tfie z-axis positively oriented from the X-ray source towards the detector plane, and being per- 
pendicular to it. The function z(t4, representing the deptfi or height w.r.t, the origin along the curve in 3D space, will 
now act as the homogeneous coordinate VV(t^, and will be used as the denominator to obtain the projection of the 
.space curve onto the detector plane, the functions .A{t^,y(t^ of the object in 3D space represent the homogeneous 

; ooordinafes X(Ui, Vtt^. The loeaJJon Mi^; /(U)) ,of ih& prajetsted 3D ouwe in the 2D image on the detector plane fe . 

• then obtained as 

[OtlSJ Therefore tha use and representation of sb bbjectshapes'by rational.Q&iercury^ has"a,physical ground in 
the imaging process. When the object is not tepresiented in a coordinate systern with z-axis aligned with the central 
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projection direcaon, a rigfd object transformation, represented by an Euclidean transformatiorras^6ut«ned>above. must 
first be applied onto the control points of x(u).y(i/),2(u) (equivalently a 4x4 matrix may be applied to the four-dimensional 
control points expressed in homogeneous coordinates). _ 

Spline curve 

[0120] Because Bezier curves that are only represented by one polynomial segment are inadequate to represent 
object shapes with high geometric detail (requiring a high degree) and because a change in shape affects all control 
points, spline curves have been introduced that allow local control of the object shape yet efficient representation. 
[0121] A p-th degree B-spllne curve is defined as 



where the {P/] represent the control points jointly fomiing the control polygon, and the {N/J^Ui] are the p-th degree 
basis spline (B-spline) basis functions defined on the non-periodic and non-uniform knot vector (m+l knots) of non- 
decreasing real numbers and with multipHcity /?f1 of a and b 

[01 22] A spline curve with no interior knots is a Bfeier curve. Such spline curve has a knot vector of the form 
t/= {0,...^b,1,...1} v>^ithmultiplKatyp+1 of 0 and t, yielding the Bernstein polynomials of degreep. 
Similar to the mapping of B6zier cun/es, the mapped spline curve can be obtained by mapping the control points 
and regenerating the splfhe ourve in the target in^e on the basis ot the mapped cbntrol points {Pf] 



. Ralional spline curve 

[01 23] A p-th degree rational spline curve is defined as 



C(w) ?=-^^_ ^ 



[01241 Again , the {Py} represent the confrol points jointly forming the control polygon, and { are the weights. When 
the knot vector is non-penodic and non-unifotm, te. 
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U = {u,....uj = {a....a.u^^„...u^.,.b....b] ------ 

wkh multiplicity jw-l of a and b, these curves are known In ttie prior art as NURBS curves (Non-Uniform Rational Basic 
Splines). A NURBS curve with no interior knots is a rational Bezier curve. In general, NURBS curves comprise non- 
rationaf B-splines and non-rational and rational Bezier curve as special cases. 

[0125] As with rational Bezier curves, all transformations and ortfiographfe projections of C(.u)can be obtained by 
applying the operation to the three-dimensional control points {P^. This property is called affine invariance. When 
perspective transfomiations are applied are applied to curves, new weights {w^ must be computed within tfie formalism 
of NURBS representation, whicti property is called invariance under perspective transfomiations. 
[01 26] Bezier and spline objects are partbularly useful to represent the outlines of diagnostic areas in medical images, 
an instantiated model of which needs to be mapped into the actual image jDriorto applying a deformation algorithm to 
obtain the actual segmentation. This mapping operation Is needed in applications such as 

• segmenting a bone on a musculoskeletal image based on deformable contour models, 
determining the lung field outlines on a thorax image or 

• determining the breast skin line on a mamraogcaphic image. 

Compound geometric object mapping 

[0127] Compound geometric objects are objects that are composed of elementary objects such as line segments, 
cjrcular arcs and polynomial shapes. The mapping of compound objects is achieved by mapping each of the constituent 
objects in turn. Examples of compound geometric objects in the field of medical imaging are the outlines of prostf^eses 
and implants used in the course, of planning a joint repfacernent in orthopedics or the outlines of screw-plates an<;f 
cages used in planning traumatologic surgery. . . - 

3. 2 Engine and User Interlace (fig. 5 and fig. 6 (a)) , ■ 

[0128] The system that implements the geometrk: mapping method consists of two parts. 

• The engine refers to the internal part that is responsible forthe chaining of the mapping of anchor points, internal 
computation of the geometric transformation, the expression and ericoding of the geometric objects with respect 
to the model and target coordinate systems, the application of the geometric transformation to the objects' defining 
paranieters and the regeneration of the objects' graphical outline in the target image. The divining oif the anchor 
point mapping rriay be based on a dependency graph of the kind that: is disclosed in Europeati patent applicatiori, 
EP A 1 349 098 for Hie mapping of measiirement points. . Anchor objects and depending geometric objects are 
representect as internal objects instantiated from classes, the design ar\d implementaUpn of whicli adhere to the 
principles of object-oriented systems, . 

Depending on the application, which will be disctosed in the sequel, the engine is further responsible for ad- 
. dltionaHasks. For example, in the case of ROI mapping and extraOlon, these tasks comprise image re-sampling 
■ .. and interpolation, irns^a rnprnory mapagecront, and database operatbns^ 

• . The user Jntertaoe constitute? the external part of the system and will typically consist of two windowis, Tiie flrajE': 
. WindQW shows the model objects, in geometric relationship to their anchor objects. The se«)n4.«^scloW 

,. represmfs the window Into wjtiich the model objects mustfjemapp^drthis window wiii typteallycornprfse ^ioiiage: 
the user interface comprises means to manipulate the position of the anchor objects iri the target irti 
g. a mouse cursor, upon whfcti the.mapped geometric objTOte are re^^ 

[pi 2SJ In a specific^bodiment the first wiridow showing ttie model objects may be omitted and may be represented, 
by a list of niodel.anchorobjects. These model £irichor objects are selected successively from the list prior to mafjping . 
. them |t>»>e .tetiget Image. . 

[0130} I7ie system w«ibasics% operate in two modes. 

-r» In an intei;active mode, the user sequeritially selecJs ttie model objects e.ig. by cfiiSdrig on ttiem, A copy of this 
. . model obiect may now be attached to the cCirsor, and tfie user is. prompted to map this model ob|ect into the.actuat .. 
.. image. ■ .... . ■ ". 

For point aricfwr ponits. the user will drag and drop each point on ft&Mended anatf^nk^ location in con- 
fomiance with the tnodel iniag& 
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compute the geometric transform. When too few points ans input than needed, the systeni^^isstie a warning 
that the transform is underdetermined. 

Anchor line objects may be defined by two points, and after selection of the model liae. each of its defining 
points is positioned in a similar way as for individual anchor points. These points serve as handles in the image 
when the line's position and orientation needs be adjusted. 

When the required number of anchor points or anchor lines is reached to compute the geometric transform, 
the system calculates the mapping between the model object coordinate system and the Image object coordinate 
system. Rnally, the model objects are mapped from the model image into the actual Image by applying the geo- 
metric transform to their defining parameters. 

Readjusting any of the anchor points in the actual Image, will invoke a re-computation of the geometric trans- 
form, and will accordingly update the position of the geometric. objects by applying the new transform to the objects' 
defining parameters. 

♦ In an automatic mode, the system highlights each of the rhodel anchor objects in turn according to the chaining 
that Is imposed on them in a dependency graph, so as to hint the user to map them in the image, and subsequently 
maps each of the objects that is defined in the coordinate system defined by the model anchor objects. The posf- 
tloning In this mode is further enhanced in that an initiat position of the subsequent points may determined on the 
basis of previous points, e.g. the Initial position ct the ttiird point in over-detemnined Euclidean mapping may be 
computed on the basis of the first two (which define an Euclidean mapping in a determined way), and the fourth 
points position may be detemiined based on the mapped position of the former three, and so on. 

f0131] Both the interactive and automatlcmode hastheadvantage that no wrong correspondence can be established. 
When e.g. a model quadrangle ABCD has to be mapped onto a tatget quadrangle A'B'C'D' the order of mapping each 
constituent object is imposed either by the user orby the system. Due to the labeling of the objects, at anytime, it is 
known which object of the model is being mapped. This method of operation Is therefore-invariant against pemiutatfohs 
of the objects to be mapped. 

3.3 Application of the object mapping methods 

Radiological scoring (fig. S) 

[0t32] The current Invention is particularfy useful in computer-assisted radiological scoring. In a preferred embodi- 
ment a set of Irriage ROIs is extracted and each ROl is juxtaposed against a set of the references images, while at ■ 
the same time scrolling through them so as to find the best matching reference stage. This principle, disclosed in 
European patent application EP A 1 293 925, may be applied to e.g. bone age scoring, v/here a fixed number of 
predefined skeletal sites need fae staged against a set of reference (mages. The anchorpoinf objects, useful to reference 
and locate all skeletal RUS arid carpal ROIs in thi Tanner aiid Whitehouse method; are e.g. the distal tip of the distal 
phalanx and the midpoint of the proximal phalahx-fhetacarpal joint of fingers III and V (4 points), ttie distal tip of the 
distal phalanx and the prdxnnal mic^soint of th6 metacarpal of the thumb (2 points), the projdmal midpoints of the 
thetacaipals If and V (two points) and the distal midpoints of the ulna and radius (two points), a total of 8 anchor pol nts. 
These points, depicted in fig 5 (a) are usedin pairs (q map RDI'sfrom a model template into the image using the 
methods of Euclidean similarity mapping outlined above. Each pair establishes the transfomi matrix that is used to 
map the ROIs of the model image into the target image. Subsequently, each ROl is extracted from the image by a re- 
sampling and intetpoTation processistich as tutfineaf or bi'^cubic s^lin^ IhteipplBtion, before it is finsU^ pta<^ in cfose^ 
>rtclnl^-«rtth ihei scorlhg reference pictufes artd r^erence Jterfis. A ecif>6matfc 'of tfie ROl images is sHbwil fn fig. 5 (b) 
.fot'theWS'l36nes;iand%, 5{c)f6rttiecatpalb6nes; ■ 
[0133] Instead df extracting ROIs from the image to be scored arid presentJng them In juxtaposed to .the reference 
pictures, an second embodiment, that uses the spatial mapping operation based on anchor points, consists in mapping 
the reference pictures into the image, in close vicinitY to the skeletel region <e.g. in.atotichfng manner to the ROlof a' 
particular region). Usingthescioli %vfteeloflheiiiouse, orthe upa^^ keys of the keypad, the refer^ce picture^ 
are£crolledthrough,andtheuser6elec^fhatFeterehc^ 

picture and skeletal region. This mannerof operiatlon has the advantage that the full input rmage remains visibte to the : 
user. 

[01^4} The ROl and reference picture mapping is furtfier advantageous in a third embodiment based on registratfon • 
where similai*/ Is measured by pixel-wise comparison based on correlation. In this embodiifientthe reference picture - . 
is mapped Into the image In a spatially coinciding manner so as to overlay on the corresponding skeletal site. The' 
anchor points with reject to whBi the ^ketetal nylons are referehcMi are used to si^ierimpose the reference picture 
onto the stel^ region to be scored. A.Pnesai' or noniflnear image regfefriMion operatidn is appfled to the ROi to find 
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its mostcorresponding position with respect to theskeletalsite and the degree of similarity of th&fegist*etiQR-ls recorded 
This registration operation is repeated for each of the reference pictures, and that stage is selected for which the degree 
of similarity is the highest. _ 

Radiological image quality control 

[0135] The current invention is further particular useful in the context of quailt/ control and quality assurance by 
means of an analyzing phantom. A phantom typically consist of a spatial arrangement of analyzing objects with fixed 
topologipal layout to measure system and image quality parameters such as SNR, MTF, DQE, contrast and geometric 
resolution. The image data analysis that is carried out to compute these parameters operates on specific imaged 
phantom locations, lb determine for example the contrast of circular disks with respect to a background, the disk area 
(i.e. some region fully contained within the disk border), and the surrounding background of the disk must be sampled. 
The spatial location of these samples may be defined in an analysis object model .(based on a CAD mode) of the 
phantom) that Is mapped into. the target image (the image of the phantom). The phantom comprises a small number' 
of easity discernible and locatable rriarker points that serve as anchor points with respect to which all analyzing objects 
are referenced. The image of these anchor points are located by the user and brought into correspondence with their 
model anchor points. A mapping tran^fonnation is computed, and the positions of all analyzing points in the image of 
the phantom are next computed. The intensity at the analyzing points' positions is finally used to compute the quality 
measures. For example, to evaluate the local contrast in (he image, the difference. of intensity in a mapped circular 
region inside a disk and a mapped annular concentric region around said disk is computed. 

Radiological image measurements^ (fig. 6) 

[01 36] The current invention is also useful to alleviate the burden of pre-positioning key measurement points In a 
radiological measurement application such as disclosed in European patent application EP A 1. 349 098. A pluraRty-of 
measurement pointy, in ttils applipation must be mapped into the image onto their corresponding anatomical location. 
Three placemerrt modes are disclosed in this patent: manual placement, ROI magnified placement and automated 
placement based on deformable contour segmentatiotT. . 

The process of the first placement rnode is enhanced by attacfting a measurement pointto the cursor aind pre.-mappfng . 
-it in th@ target image onto a location that is obtained by applying the established geometric tFansfonn to' eacEi model 
measurement point of the measurement template. Anchor points are used to establish the geometric transfomr, how- 
ever also, measurernejjt poiritstheniselves may be used to refine, at^ualize or compute the transfomiatidJi'On the ffy 
as the points' positions become available in the course of executing the measurement template. 
The second placement mode is speeded up by the current invention by first auto-zooming the ROI that comprises a 
measurement point to a larger or full resolution, and secondly by pre-mapping the measurement points towards an- 
initial position in close vicinity to their true anatomical position. The auto-zooming step magnifies a particular portion: 
of the image either in a separate popped-tip window, of in ^ manner so as to completely flit the current image windows 
The pre-mapfjing stetJ maps a specific measurernent pQint towards Bs- initial position te'its.associated ROI; Ihe user • 
accomplishes the mapping task by dragging the point towards.its finatposifipn in said magnified ROI . The coordinates; 
oif jtfie^ointta ttie oflginatjniage are flnal.i^.cqiTipiuled.fc@r:applying.the.inye^ that links the coof- ■ 

diiiate system'oif t^e.zobi^$d RQI «itli ittiat pf tiie arig|n,ari.mage.;BothiROl^etect|Qn andmeaabHrement point rapping :, 
are faaged on the methods.of anciioroi5jei$^.Qpfresppf^ and geekmetrie fransfomnation computatfon arid applicatfonf; 

. ioii(t(ined.abQve.,, ; „;.. ; -.^ - ' ; .-n- . .. i'.,.^ -.i^-. . .V: - 

Th^ ttii^f ja^riiem.tiiode,;^^^ 
ti^t^p^.tt^;ppsl»PQ 0f;m^^^ 

gpMn€^,#nd phptpttij^to^ subsequently . deformed model contour. -The- telfiaJ.p<»e .. ■. 

(r^iion, trapslatiqti at!d 

spondence' of whfch witti tfie target andior objects establishes the 

£0;(3.7] Particular^ vy|t.eapqtfoiinihg 3^ measurgme.cit.templates according to Euiropean patent application EP A 1 v 
34;9 098., ilefined:a§a |l^^ image and. :; 

di?p%edas.a staokin ti^;template.wind6w,:^e^urerrjet^ 

tfi^ jpotTesp)ndiogisii^^ itr tfte actual 30. image. High-resoluttan 3D. irnage of current- rirodaJities, such as MR or GT 
t^ically comprise iOQQ sri^^ 

icrpiltijfough thestsuik'^ijtees utr^ positioned^ 
% uslog a geometric tfaBsfonriatfw .e^^tofelii^ ^ <»fr^paedence of a few. nurrt>er k ^xeitna^^ anchor points in 
mptiel jahd actual 30, image,. thf number jQi .sQpfes to Sf^-dJ th«oyai«"(?an t)%n?iww ff^raBge. specific to the ? , 

measurement point, tfiereby Fedudng the search tim6. For the transfonnalion fs three dimensional, not only a initiafi. - 
sHe^ js prerselef^dwhen nu^qwiga |»lnt, is 9lso ^fe-posttioh£»l withln.a siii^. Ailet hadng^^leel^ttie.target slic&i. , 
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the point is further positioned witliln thff slice by dragging it onto its associated anatomical position .-The 3© coordinates 
of all measurement points are subsequently used to compute 3D metric entities sucti as distances and angles. 

Ortiwpedic and traumatologic implant templating 

[0138] Instead of mapping atomic geometric objects such as points, lines, circles and splines, objects composed of 
such elementary objects may be mapped as a whole from a model image window into the target image, solely on the 
■ basis of a small number of anchor points, by applying the mapping transformation to each of the constituent elementary 
objects. Such composed objects are typically implant or prosthesis outlines obtained directly from a prosthesis man- 
ufacturer as a 2D or 3D CAD model or from vectorized scanned corttours of a conventional 2D acetate template. This 
mapping considerably speeds up the selection and placement of the prosthesis, a process commonly known as tem- 
plating. When perfomied on film or on a computer display as the non-automated computer equivalent of manual film 
tempiating, it requires 6)rtensive manipulation of the objects, such as successive translations and rotations in a non- 
systematic manner so as to have a fit of the prosthesis outline with the anatomic image data. The current invention fe 
particularly suited to aid in the positioning process by providing an initial position that may coincide well with the par- 
ticular imaged patient anatomy, the accuracy of which depends on the positional accuracy of the anchor points or 
objects and the accuracy with which the template is defined in the coordinate system of the anchor objects. There is 
also a positional dependency of template objects that may be used to automate the placement of further objects. In 
total hip replacement, the planning comprises the selection and placement of typically three components of an implant: 
a femoral stem, a femoral head and an acetabular cup. When the femoral stem is first placed in the radiographic image, 
the placement of the femoral head, which mechanicaify fits onto the stem, may be auto-placed, using the femoral stem 
as the anchor object, such that the axis of the head coincides with the femoral neck axis of the femoral template 
oomponent. Lilcewiae the acetabular cup is auto-placed by a mapping that now uses the femoral head as the anchor 
object: th© translational component follows from the position of. the femoral head center and it remains to apply a 
suitable rotation to the cup. When the acetabular cup is first placed, its center and orientation serve to determine the 
prfiposWon and -orientation of the femoral head, which in turn detemiin e the preposition'and -orientation of the femoral 
stent. 

[01391 Another application that )s related to prosthesis tempiating is In the field of traumatology, where the planning 
of surgical procedures requires the plaeement of graphical models of screw cages onto the iriiage- Also here, this 
placement may be speeded up and made more accurate when the graphical models are linked to and made dependent 
on the prior placement of anchor points or anchor objects. Whenever the position of the ancestor objects is known, 
they are used as anchor objects to which the depending objects are referred and pre-pbsitionecf by the mapping inetft- 
ods as described in the above. 

Radiotogicalbnage.segmeatation. > - 

f0140I The curtent invention is also useful in the field= Image segmentation. Fdr medical images, this basic lmage, 
processing operation aims at delineating anatomic strxictiiresahd organs, in ordertoperfomi e.g. planim perimeter, 
area or volumetric measurements. For this purpose, graphic models are used that iriitiaiize the outline of theobject(s> 
to besegmented. These contours are subsequently deftirmedby applying ima^intemal (e.g. gradieflts) andcphtduf- 

' fntemat (e.g;loeaf curvature) forces, an approach^nown in the prior art asi«n'aRes seigNientatton. However.'aft importanl 
problem Involves the pre^positioning of the model contours. By defining the cont(Sur(s) in the coordinate system of a 
small nurr^ber of anchor objects (e.g anatomk^l landmark points), the contour<s]' initial position in the target image' 

. can'bftcoimputBdby applying the estab8stecl|fedtnetric trsmsforrnirtkjn tffftie contour's <lefi^ ex- 
ampie, Bpbljrgonat oiitBiie&an be me[{)ped by applying the trarisfbrtmSentiQ^^^ outline 
can.be nia^etfby Applying thetransfAfmationtoThe Ifi either example; the inltlai bohtoui-otrtllne 

In tfie tarigei. Image is gener^ed on the basis of the defining paraniteters. Examples of &pplicatfons; of segfhenfetiorr 
are given In the sequel. - ' ' ' 

[0141 J In the field of mammography, computer-aided diagnosis algorithms rriust operate in the image on the tsreafet 
mass region only; hence a aegmentation of the skin line is needed; A mapping of ah initial model contourmay be'based 
on characteristic points descfibfng the breast outline. In parHeular, the Jbllowing ahchor jpblnta may ijsed in affihe^ 
warping atnodel outiinetowards an initial breast skin oufline:ia)1n the CC(cmnio-caudal):mammographb view: nipple; 
left and right point of the breast irriage near the chest- walfi .(b> in ttW ML6 (niedto-iaW^ oblique) view: base of the 

.:^ecteraliniiscte at tife chest 8fe outerrnost position of th^ pie<^ral fhuscle ait th&top of the breast image, the ' 
Vil^le-arid thie-bott(»n of btes^t iriiage near the chest wait Th^hi^l^ed1rfiM<»3nt6ur is fi^^ according' 

. to e.l|.'shsife6S £y9pr6aeh. Until it coMcfe^ with the actual % region* boundary irt the" 

. .mafmnogrdrTK ■ ' • ' 

f{6l 42}; Similar region delineation is neeiiad in the computer-assisted diagt\osis ot tfibi^ bhages, ivhere'fiie exteitr . 
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of lung fielcb need be determined. The mapping of lung field outlines here may be based oh etefaGteristJC points in a 
thorax image. In particular, the following anchor points may be used in affinely or projectively mapping a model outline 
towards an initial lung field outline: left and right costophrenic sulcus (also called the costophrenic angles, each angle 
formed between the inner surface of the ribs and the dome of the diaphragm) . left and right Intersection of the heart 
shadow with the dome of the diaphragm, and left and right lower clavicle-"rib three" intersection points. 



Claims 

1. A method of mapping geometric objects in a digital target image characterized by 

selecting at least one anchor object part of a model comprising at least one geometric object, 
mapping said anchor object into said target image so as to obtain a mapped anchor object, 

- computing a geometric transfomiation on the basis of at feast one corresportdence between an anchor object ' 
and a corresponding mapped anchor object, 

- applying the calculated geometric transformation to at feast one geometric object, ericoded with respect to 
said anchor objects, thereby mapptng^ the geometric object from said model to d correspoodlng mapped po- 
sition in said target image. 

2. 'A method according to claim 1 whereitr said nnodel is displayed as a model image comprising anchor objects. 

3. A method according to any of the preceding claims wherein said anchor objects are points. 

4. A method according to any of the preceding claims wherein said anchor objects are lines. 

5. A method according to claim 2 wherein said geometric objects are rendered in said model tnage. 

. A method according to claim 5 wherein said geometric objects are encoded relative to said anchor objects. 

. A method according to any of the preceding claims wherein said anchor objects and/or said geometrical objects 
represent anatomic entities. 

, A method according to any of the preceding claims wherein said target image is a medical image. 
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FIG. 1 
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FIG. 2 
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FIG. 3 (b) 
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Fig. 5 (b). RUS (Radius, Ulna, Short Bones) Regions of Interest 
Distal Radius and Ulna Regions of Interest 
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Fig. 5 (c). Carpal Regions Of Interest 
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Select anchor objects in 
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Encoded geometric 
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Apply geometric transform to encoded 
objects to be mapped 
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